Check that precision of `extension_api.json` matches build options #1714

dsnopek · 2025-02-21T20:19:15Z

We regularly get bug reports from folks who compile their GDExtension with precision=double, but use an extension_api.json from a single precision build of Godot.

This PR uses an additonal field that godotengine/godot#103137 adds to the extension_api.json to validate that the file matches the build options

Marking as DRAFT until the Godot PR is merged

(It should be fine to merge this after the Godot PR is merged, but even before it's in a stable release of Godot, because it won't do anything if the new field is missing from the extension_api.json)

dsnopek · 2025-02-21T20:20:39Z

@enetheru I'll eventually need your help figuring out the best way to do this in CMake. Maybe actually this check should be moved into binding_generator.py since then it's already on the Python side and CMake doesn't need any additional changes?

enetheru · 2025-02-22T00:03:13Z

@dsnopek CMake gained the ability to parse json in 3.19, we are on 3.17 https://cmake.org/cmake/help/latest/command/string.html#json

Looks simple enough to use, if we are ok with the cmake version bump.

tools/godotcpp.py

dsnopek · 2025-02-27T19:30:56Z

CMake gained the ability to parse json in 3.19, we are on 3.17 https://cmake.org/cmake/help/latest/command/string.html#json

Looks simple enough to use, if we are ok with the cmake version bump.

We're already using Python, and have the data available in Python, so I just moved the check into binding_generator.py which I think means this should just work for CMake now too :-)

enetheru · 2025-03-01T03:30:50Z

How do I go about testing this?
Do I need to build the godot with #103137 and dump a json? to manifest the error?

dsnopek · 2025-03-01T03:40:03Z

How do I go about testing this? Do I need to build the godot with #103137 and dump a json? to manifest the error?

That would work, although, an easier way to test is just to add "presion": "double" to the header section of the extension_api.json and then build with single precision (or vice versa)

enetheru · 2025-03-01T04:21:25Z

OK, that was much easier, I can confirm that a build will fail using cmake with an error message that is legible enough to understand the problem, even though it looks a bit garbled.

Would be nicer to fail during the configure phase rather than the build phase in case of larger projects that compile a bunch of stuff prior to godot-cpp. I'll see if i can whip something up with cmake 3.19 quickly being able to parse json directly with cmake might come in handy in the future.

Our CI's are already updated to Ubuntu 22.0.4 which has a CMake version of 3.22.1
Using https://pkgs.org/download/cmake tells me that only Debian 11(Bullseye) is lower at 3.18.4.
https://endoflife.date/debian for LTS of Debian 11 is (31 Aug 2026)

MinGW64 + Ninja

[1/79] Generating bindings
FAILED: gen/include/godot_cpp/core/ext_wrappers.gen.inc gen/include/godot_cpp/core/gdvirtual.gen.inc <snipped>
C:\WINDOWS\system32\cmd.exe /C "cd /D C:\Godot\src\godot-cpp && C:\Users\nicho\AppData\Local\Programs\Python\Python313\python.exe -c "from binding_generator import generate_bindings;generate_bindings( api_filepath='C:/Godot/src/godot-cpp/cmake-build-release-clion-mingw/extension_api.json', use_template_get_node='True', bits='64', precision='', output_dir='C:/Godot/src/godot-cpp/cmake-build-release-clion-mingw')""
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    from binding_generator import generate_bindings;generate_bindings( api_filepath='C:/Godot/src/godot-cpp/cmake-build-release-clion-mingw/extension_api.json', use_template_get_node='True', bits='64', precision='', output_dir='C:/Godot/src/godot-cpp/cmake-build-release-clion-mingw')
                                                    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Godot\src\godot-cpp\binding_generator.py", line 281, in generate_bindings
    _generate_bindings(api, api_filepath, use_template_get_node, bits, precision, output_dir)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Godot\src\godot-cpp\binding_generator.py", line 286, in _generate_bindings
    raise Exception(
        f"Cannot do a precision={precision} build using '{api_filepath}' which was generated by Godot built with precision={api['header']['precision']}"
    )
Exception: Cannot do a precision= build using 'C:/Godot/src/godot-cpp/cmake-build-release-clion-mingw/extension_api.json' which was generated by Godot built with precision=double
ninja: build stopped: subcommand failed.

msvc

====================[ Build | godot-cpp.editor | Release-Visual Studio ]========
"C:\Program Files\CMake\bin\cmake.exe" --build C:\Godot\src\godot-cpp\cmake-build-release-visual-studio --target godot-cpp.editor --config Release --config Release
MSBuild version 17.13.15+18b3035f6 for .NET Framework

  Checking File Globs
  1>Checking Build System
  Generating bindings
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
      from binding_generator import generate_bindings;generate_bindings( api_filepath='C:/Godot/src/godot-cpp/cmake-build-release-visual-studio/extension_api.json', use_template_get_node='True', bits='64', precision='', output_dir='C:/Godot/src/godot-cpp/cmake-build-release-visual-studio')
                                                      ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Godot\src\godot-cpp\binding_generator.py", line 281, in generate_bindings
      _generate_bindings(api, api_filepath, use_template_get_node, bits, precision, output_dir)
      ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Godot\src\godot-cpp\binding_generator.py", line 286, in _generate_bindings
      raise Exception(
          f"Cannot do a precision={precision} build using '{api_filepath}' which was generated by Godot built with precision={api['header']['precision']}"
      )
  Exception: Cannot do a precision= build using 'C:/Godot/src/godot-cpp/cmake-build-release-visual-studio/extension_api.json' which was generated by Godot built with precision=double
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\Godot\src\godot-cpp\cmake-build-release-visual-studio\extension_api.json;C:\Godot\src\godot-cpp\CMakeLists.txt' exited with code 1. [C:\Godot\src\godot-cpp\cmake-build-release-visual-studio\godot-cpp.editor.vcxproj]

enetheru · 2025-03-01T04:57:04Z

If the API contains the precision and its invalid to compile with a differing result, then perhaps source the precision directly from the API and remove it as an option from the compile commands?

Anyway, this is a quick draft of something that fails at configure time with context at the top, in godot-cpp.cmake at roughly line 223

    # API json File
    set(GODOT_GDEXTENSION_API_FILE "${GODOT_GDEXTENSION_DIR}/extension_api.json")
    if( GODOT_CUSTOM_API_FILE )  # User-defined override.
        set(GODOT_GDEXTENSION_API_FILE "${GODOT_CUSTOM_API_FILE}")
    endif()

    # Check for correct precision
    file(READ ${GODOT_GDEXTENSION_API_FILE} API_JSON )
    string(JSON API_PRECISION ERROR_VARIABLE API_ERROR GET "${API_JSON}" "header" "precision")
    if( API_ERROR )
        message(WARNING "In file '${GODOT_GDEXTENSION_API_FILE}': \"precision\" is missing from the header object."
                " It must have been created prior to 4.4, continuing with the assumption you are using the correct api file."
                " See https://github.com/godotengine/godot-cpp/pull/1714 for more information.")
    elseif( NOT API_PRECISION STREQUAL GODOT_PRECISION )
        message( FATAL_ERROR "Cannot do a precision=${GODOT_PRECISION} build using '${GODOT_GDEXTENSION_API_FILE}'"
                " which was generated by Godot built with precision=${API_PRECISION}" )
    endif()

Edited to make it a little nicer.

dsnopek · 2025-03-01T17:07:28Z

@enetheru Thanks for testing and investigation!

I can confirm that a build will fail using cmake with an error message that is legible enough to understand the problem, even though it looks a bit garbled.

That's about on par with the current message from SCons. It was also somewhat more legible in my first SCons implementation that used UserError in the SConstruct file, rather than Exception in the Python script.

But I personally think this is fine for now, if it gets us better compatibility - we can always improve it later. I'd be curious to hear what folks with more SCons knowledge think of this, though.

If the API contains the precision and its invalid to compile with a differing result, then perhaps source the precision directly from the API and remove it as an option from the compile commands?

I'm not crazy about this idea, because it doesn't explain to the user why the option isn't available. Although, I suppose it could be paired with a warning message when the logic is deciding to remove the option?

Anyway, this is a quick draft of something that fails at configure time with context at the top, in godot-cpp.cmake at roughly line 223

Thanks! What does the error message look like with that implementation?

enetheru · 2025-03-01T22:47:07Z

Thanks! What does the error message look like with that implementation?

With older api.json without the header tag and lets you continue with a warning:

CMake Warning at cmake/godotcpp.cmake:233 (message):
  In file 'gdextension/extension_api.json': "precision" is missing from the
  header object.  It must have been created prior to 4.4, continuing with the
  assumption you are using the correct api file.  See
  https://github.com/godotengine/godot-cpp/pull/1714 for more information.
Call Stack (most recent call first):
  CMakeLists.txt:52 (godotcpp_generate)

With a json file with mismatching header.precision that fails at configure time:

CMake Error at cmake/godotcpp.cmake:237 (message):
  Cannot do a precision=single build using 'gdextension/extension_api.json'
  which was generated by Godot built with precision=double
Call Stack (most recent call first):
  CMakeLists.txt:52 (godotcpp_generate)

-- Configuring incomplete, errors occurred!

Repiteo · 2025-03-07T21:21:16Z

Godot PR merged, this is now ready for review!

dsnopek added the enhancement This is an enhancement on the current functionality label Feb 21, 2025

dsnopek added this to the 4.x milestone Feb 21, 2025

dsnopek requested a review from a team as a code owner February 21, 2025 20:19

dsnopek marked this pull request as draft February 21, 2025 20:20

dsnopek mentioned this pull request Feb 21, 2025

GDExtension: Include precision in extension_api.json godotengine/godot#103137

Merged

touilleMan reviewed Feb 22, 2025

View reviewed changes

tools/godotcpp.py Outdated Show resolved Hide resolved

touilleMan approved these changes Feb 22, 2025

View reviewed changes

Check that precision of extension_api.json matches build options

0a73df5

dsnopek force-pushed the extension-api-precision branch from 93b484e to 0a73df5 Compare February 27, 2025 19:28

dsnopek added cherrypick:4.3 cherrypick:4.4 labels Feb 28, 2025

Repiteo marked this pull request as ready for review March 7, 2025 21:21

Repiteo changed the title ~~[DRAFT] Check that precision of extension_api.json matches build options~~ Check that precision of extension_api.json matches build options Mar 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check that precision of `extension_api.json` matches build options #1714

Check that precision of `extension_api.json` matches build options #1714

dsnopek commented Feb 21, 2025

dsnopek commented Feb 21, 2025

enetheru commented Feb 22, 2025

dsnopek commented Feb 27, 2025

enetheru commented Mar 1, 2025

dsnopek commented Mar 1, 2025

enetheru commented Mar 1, 2025

enetheru commented Mar 1, 2025 •

edited

Loading

dsnopek commented Mar 1, 2025

enetheru commented Mar 1, 2025

Repiteo commented Mar 7, 2025

Check that precision of extension_api.json matches build options #1714

Are you sure you want to change the base?

Check that precision of extension_api.json matches build options #1714

Conversation

dsnopek commented Feb 21, 2025

dsnopek commented Feb 21, 2025

enetheru commented Feb 22, 2025

dsnopek commented Feb 27, 2025

enetheru commented Mar 1, 2025

dsnopek commented Mar 1, 2025

enetheru commented Mar 1, 2025

enetheru commented Mar 1, 2025 • edited Loading

dsnopek commented Mar 1, 2025

enetheru commented Mar 1, 2025

Repiteo commented Mar 7, 2025

Check that precision of `extension_api.json` matches build options #1714

Check that precision of `extension_api.json` matches build options #1714

enetheru commented Mar 1, 2025 •

edited

Loading